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ABSTRACT 


This volume I Measures of Efiectiveness i describes the development 
of both quantitative and qualitative criteria that were used) to evalu- 
ateuconceptional systems for automating the functions for the FBI 
Ideiitification Divisionn Specific alternative systems for automation 
we r| compared by using these developed criteria > defined as Measures 
Of Effectiveness (M0E)» to gauge system's perfottnance in attempting to 
achieve certain goals. The MOE> essentially measurement tools that 
were developed through the combination of suitable parameters > pertain 
to each conceivable area of system operation. The methods and 
approaches used, both in selecting the parameters and in using the 
resulting MOE, are described. For a synopsis of the entire report, 
see the Executive Summary in the Compendium (Volume I). 
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SECTION I 


INTRODUCTION 


A. DEFINITION 

Measuret of effectiveness (MOE) are the parameters used to gauge 
the state and performance of a system in attempting to achieve its 
goals. 


In this document I system is taken to mean people^ equipment > 
schemes, and interfaces. The management is the entity in control of 
the system. 


B. OBJECTIVES 

The purpose of this document is to develop measures that are 
significant in evaluating alternative designs for the automation of 
the FBI Identification Division functions. 

It is also the intent of this document to indicate, where 
necessary, methods and approaches to be used in the course of applying 
these measures . 


C. SCOPE 

This document covers qualitative and quantitative measures of 
effectiveness of alternative designs for the automation of functions 
specified in the Top Down Functional Analysis (TDFA) (Reference 1). 
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SECTION II 


GROUPING or FEASIBn<ITY STUDY HEASURES 

Th« NOE par«iiefceri are preaeribed to match the four top-level 
functionc outlined in the TDFA# There are: 

(1) Provide fingerprint identification. 

(2) Provide record keeping services. 

(3) Hanagement of the data base(s). 

(A) Executive direction and control. 

The MOE parasm!ter| are grouped into two suijor categories: 

(1) Qualitative parameters that can^ for the most part, be ^ 
expressed in terms of distinctive properties such as well 
developed, insufficient etc. 

(2) Quantitative parameters that can be expressed in terms of 
measurable units. This category has two subcategories: 

(a) Parameters whose determination is within study scope. 

(b) Parameters whose determination is outside study scope. 

A list of all sieasures of effectiveness by category or 
subcategory are listed in Figure 2-1, 



DETaiMIlsiATION OUTSIDE SIUDV SCOPE I DETERMimTION WlTHiy SKIDY SCOPE 


quantitative 


THROUOHfUT AND HESTON .. TIME 



MAINTAINABILITY 


MLIABILITY 


GPEHABIUIY 


AVAILABILITY 


OBSERVABILITY 


UTILIZATION 


FLEXIBILITY 


ACCURACY AND COMPLETENESS 


INTEGRITY 


SICURlTY 


Figure 2-1. Parameters for Measures of Effectiveness 
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SECTION III 


DEVELOPMENT AND APPLICATION OF MEASURES OF EFFECTIVENESS 


A. INTRODUCTION 

A clear and definitive diitinction between the MOE and the 
functional requirenenta ahould be made at the outaet> 

While the functional requirementa ipecify the capabilities that 
ahould be incorporated into a design^ the MOE address the parameters 
that can test these capabilities. The MOE are not meant to reveal 
deficiencies in designs because of a missing function but are designed 
to provide a measurement tool to the functional requirements in 
determining system design capabilities. As suchf MOE are considered 
complementary to the functiotrSl requirements. 

The MOE were developed in a generalised format in order to 
facilitate their application to alternative designs. However, their 
general framework is the Identification D>ivision functions. 

Two assumptions were made when the MOE were developed t 

(1) One of the alternative designs will be implenKnted 
sometime in the 20th Century. During that time the 
external and the internal environment will not change the 
basic Identification Division functions from either the 
current system environment or as they are stated in the 
Top Down Functional Analysis d ocument. 

(2) All alternative designs will involve a docuswnt and/or 
data flow system made up of a network of stages somehow 
linked together in order to accomplish the identification 
and record keeping functions. 


B. QUALITATIVE MEASURES 
1. Maintainability 

Maintainability is a design and implementation attribute that is 
related to restoration of components or subsystems to operational 
status as a result of maintenance procedures and resources. 

The objectives of maintainability are to: 

(1) Decrease system complexity and maintenance duration. 

(2) Decrease equipment maintenance cost, 

y (3) Provide positive fault isolation. 

(4) Increase equipment up-time. 
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The tyiteii deeignec ie expected te pecfora the foDlowinf at the 
outset t 

(1) Generate Mintainabllity requirenents at systeB and 
subsystee levels • 

(2) Develop nethodology to control suiintainabili.ty variables* 

(3) Specify a definitive plan to develop a departsMint capable 
of carrying out the functions of the auiintainability 
prograsi* 

The basic elesMnt of siaintsinability is the unit of tine* It is 
used to noasure naintainability factors such as: 

(1) Equipnent design characteristics* This includes physical 
aspects^ testingi tool requirenent8> and skill level 
needed « 

(2) Maintenance personnel skill level, experience, and 
technical proficiency. 

(3) Logistics and maintenance organization support iiivolved 
in maintaining the system. 

System design features required to optimize maintainabili;*;y 
include: 

(1) Quick and positive recognition of equipnent malfunction. 

(2) Quick and positive identification of defective components 

(3) Available maintenance skills and training to develop 
adequate proficiency. 

(4) Optimum aneessibility to equipment. 

(5) Low mean time to perform maintenance. 


2. Operability 

■'ll 

Operability focuses on the design aspects of the system pertain- 
ing to the intersictions among the system, the support persoi:;hel, the 
management, the physical environment, and the current system during 
and after implementation* This is necessary to maintain production, 
qualitatively and quantitatively, at the required level. 

The items to be considered include level of difficulty in 
directing system operations, operatianal control model, man-machine 
interfaces, technical support, coexistence of parallel operations, and 
transition from the=.jurrent to an autoisated system. 
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• * Level of Difficulty in Directing System Operations . The 
level c£ difficulty in direcMng syeteii operations should be viewed in 
terns of functions performed by the FBI personnel during systen start 
up^ nornal operations , degraded mode operations, shutdown* and after a 
crash* 


b. Operational Control Model . System design is expected to 
provide Identification Division management with techniques and proce- 
dures to predict the environment. An operation model to determine 
allocation of resources (people, space, equipment, money) in order to 
maintain or to attain certain production level at a specific response 
time is also expected. The level of detail should respond to questions 
such as these: 

(1) How many continuous processing hours are required from 
each work station to handle a specific workload? 

(2) How many employee-hours, at a certain grade, should be 
scheduled to keep a work station hourly output at a 
nominal rate? 

(3) /( How to man an assembly line work station 15 continuous 

hours a day? 

(4) What will operators do if subsystems crash? Will the "manage 
ment tolerate 300 idle operators for 5 hours once a week? 


c. Man-Machine Interfaces . Man-machine interfaces should 
provide work space arrangement of elements, components, and subsystems 
that maximize operator motor skills. Interfaces should use several of 
the operator’s sensor channels, especially during critical periods of 
system operation. Display design, shape, size, and color should aim 
toward efficient mental processing. System support personnel should 
be optimally allocated to handle daily workloads by type. 


d. T echnical Support . Phasing in automation will result in 
the reductioiT of operating personnel as a production component in the 
overall process . Ttie functions that the human continues to perform 
are as a decision maker, detector of irregularity, and troubleshooter. 

The automated system will need increased technical support. The 
technical support needed for a production operation such as the 
Identification Division is a mix of the following skills: Industrial 

engineering/operation research, computer science (hardware, software 
and data base management), programming, and maintenance support. 


e . Coexistence of Parallel Operations?(Current and 
Automated) . System design should include thA total system operation. 
If the current system is to stay in operation for years to come, or 


o 


even i£ it is to be the backup in case the autonated system is 
aborted, then the automated system should be compatible; s^'ith the 
current system in the following areas: 

(1) Nomenclature used in both systems should be clear to 
employees in both systems who may be involved in the 
interface. 

(2) Allocation of employees to both systems shouXci be 
based on response time requirements and costs. 

(3) The design should impose response time requirements 
when documents move from one system to another. 

(4) The .interfaces between the two systems should be 
desi||hed to optimize operations. The exit point of 
one system should correspond to an entry point of 
another. 

(5) The transportation mode between the tw<v systems 
should be compatible. 

(6) ’Employees in charge of the interfaces should be 
familiar with the operation of both systems. 

(7) Audit trails for exchanges of documents should be 

maintained and criteria for exchanges should be 
defined. v ' , 


f. Transition from the Current to an Automated System . 
Transition 'should use diagrams, sketches, networks, and bar charts t6 
indicate the dates of various; transitions by subsystems and the 
duration of each transitiii^n. The layout of subsystem components for 
each phase should be described. Work load shifts, by type, from the 
current to an automated system should be determined along with 
personnel requirements (number, skill levels and mix) for each 
transition phase. Consideration will have to be given to noise 
introduced with each phase. For example, the AIDS III technical file 
conversion had an adverse impact on fingerprint processing response 
time in the manual system. Similar events need to be anticipated and 
dealt with through reasonable projections and planning. 


3. Observability --j 

While operability is the active measure of system operation, 
observability is the passive measure that closes the loop in an 
effective control system. 

A system is said to be observable if it can produce measurements 
at different stages and times that contain sufficient information to 
enable the complete identification of the system and workload status. 
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Two ca(;e;?:orie> of observability are identified: aanagement 

parasieters and system control parameters. 

i( 

a. Management Parameters . Periodic status reports including 
workload throughput by typei turnaround time of documents by type, 
operator performance, system availability, and cost per transaction 
per week or month. 


b. System Control Parameters . These parameters include: 

(1) Subsystem status such as operational modes (batch, 
real-time) and non-operational conditions (down - awaiting 
maintenance, down - being maintained, and the expected 
time to restoration). 

(2) System configuration status. This item pertains to the 
information relative to the number and configuration of 
components and subsystems that are functioning in a normal’ 
mode at any point in time. The level of details should be 
sufficient to enable operators and managers to effect 
decisions relative to the operability of^the system. 

(3) Observable parameters. The par^Jmeters that assist in 

identifying process control status at various work 
stations or subsystems are: arrival rate, queues, 

transactions- waiting after being processed, service times, 
response times, transaction status and location, and 
turnaround time per subsystem and system per transaction 
type. 


4. Flexibility ' ' 

Flexibility points to a design feature that will allow system 
additions, upgrading, and maintenance in order to achieve the 
following results: 

(1) Keep maintenance cost at a reasonable level. 

.1 

(2) Avoid major redesign of software, hardware, and 
communication. 

(3) Reduce the possibility of near-term obsolescence. Two 
features of this measure are standardization of components 
and modularity of subsystems. 

5. Integrity 

Integrity is|a measure of the procedures utilized to ensure that 
data and information throughout the system will remain intact and 
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accessible after starts^ termination^ crashes > transmissions^ file 
updates t manipulation t input i and output. 


6. Security 

This measure evaluates techniques embedded in system designs to 

ensure that no unauthorized person or agency gains access to data, 
documents, or files, either physically or by using electronic or 
photographic devices. 

'll 

/ ^ ^ 

C. QUANTITATIVE MEASURE CDET ERJn NATION OUTSIDE STUDY SCOPE) 

1. Response Time Benefit 

' It is highly probable that the identification system users 

derive significant benefit increases from improved system response 
time. The problem is that the quantitative relationship(s) between 
response time and benefits are not known. Therefore, determining an 
absolute cost/benefit ratio for alternative systems is not feasible 
within this study scope. Instead, two^Other approaches similar to 
cost/benefit ratio are suggested: a.)' equal response time approach, 

and b.) response cost approach. 

a. Equal Response Time Approach . This approach postulates 
that system design alternatives can be compared on the basis pf cost 
alone if their respective response times are equal. 

Assume that the system user benefits attributable to response 
time are represented by a single-valued decreasing function of 
response time: 


B « f(/lT) 

o<4T<k 

(Equation 1) 

where 


o 

B “ benefits in dollars 


'b 


^T * weighted response time (as in Equation 6) 

i,)' 

k * required response .time threshold such as 4, 8, 48, or 
96 hours v\ '' 


Graphically that function may be represented by the curve shown in 
Figure 3-1. 

If there are n alternatives and Cj| is designated to be the 
total cost (labor; capital equipment; maintenance; leasing; and 
facilities) associated with alternative Aj,; then the cost benefit 
ratio is equal to: 


or 




(Equation 2) 


Then comparing alternatives against a bench mark base case: 


/ _ b ^n-1 

KdlT f?dTT_ , , 

— ~C * ^ ^Equation 3) 

will result in percent difference (in-i) between alternative An-1 
and the base case A^. Cb and F(dT)b are the cost and benefits of 
the base case. 




ts 

c 

MJ 

z 

Ui 

e 





Figure 3 -Jl. Response Time Versus Benefits in Dollars 





If enough flexibility resides in each alternative design* 
production capacities can be modified to produce equal response times 
for all the alternatives: 


dTj «• dTj - dXj ... - 


which will result in having the benefits equal: 

B(dT)j - B(dT>2 - B(dT>3 ... - B(dT)^ 


f 


J 

! 


The "new" cost associated with din modification can be 
generated based on the modified capacities. 

Since the denominators in the cost benefit ratio of all 
alternatives become equal* then the percent difference between 
alternatives In-i of Equation 3 is reduced to the cost variables 
only: 


I , — X 100 for k hours (Equation 4) 

n-1 G. .1 ■ 


o Different In-i values may be computed for different response 
.times. The results can be summarized in a table similar to Table 
3-1. Graphical representation such as Figure 3-2 of In-i values 
versus dT for each alternative could be helpful in revealing 
interesting points of intersection. Similar plots covering a period of 
years could be attempted in order to test possibilities of cost/benefit 
reversals due to system degradation because of workload or other 
factors. 

The equal response time approach is feasible if simulation 
techniques are used under similar workload conditions. All 
idiosyncrasies of alternatives should be either eliminated or added 
equally to each alternative. 

b) Response Cost Approach . This approach is a direct application 
of costs and weighted response time of alternatives. No modification 
to capacities are required. Each alternative will have a response 
cost equal to: 



C 


n 

Xlry 


for AT^ * K hours^ (Equation 5) 
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Table 3~1. Cost/Benefit Comparisons for 
Different Response Times 



Ti 

T2 

Tk 

Ai 

Il,l 

Il,2 

--- Il,k 

^2 

^2,1 

l2,2 

I2,ic 

^-l 

— "r~ 

^n-1,1 

^n^l,2 

- ^ ^n-l,k 


where ^IT is obtained from Equation 6* The units of will be in 

terms of dollars per hour> Comparisons of values of for all 
alternatives ca^ be done in a manner similar to the equal response 
time approach. 




■■j 



Figure 3-2. Cost/Benefit Ratios for Alternatives 
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The advantage of reapokie coat approach la that it does not add 
a layer of approximation ^ by using simulation) as the equal response 
time approach does. 

!/ 

Parameters such as cost per transaction) or annual cost per 
document type are considered subsets of Response Cost Approach. 


D. QUANTITATIVE MEASURES (DETERMINATION WITHIN STUDY SCOPE) 

(( 

1. Throughput Slid Response Time II 

a. Definitions . Throughput volume is the total number of 
documents by type that are completely processed by the system during a 
specific period of time called response time. 


Response time is the total' time interval between the instant a 
transaction transmitted to the Identification Division to the time 
a system res^hnse reaches its point of impact. 

b. Discussion. The throughput is the outcome of a workload 
mix composed of the following type of requests. 

(1) Expedite. 

(2) Criminal Fingerprint. 


(3) Applicant Fingerprint. 

(4) Disposition notices. 

(5) Miscellaneous. 

For the purpose of this document, the response time is broken 

down intb six disC^inct, non-overlapping segments: 

,, 

All ■ Time interval from the instant a transaction is 

originate^ to the instant it reaches Identification 
Division. \ 

m Time Interval from the time a transaction reaches 

the Identificaripn Division to the time it enters 
the system for process irig. 

^T 3 ■ Time interval for complete incernal system 

processing^ i.e.) entry time to response generation 
time. 

AT^ ■ Time interval from the time a response is generated 

to the time it is transmitted. 

^T 5 > Time interval from the time a response is 

transmitted to the tiiae it reaches its point of 
origin. 


3-10 


■ Tiae interval from the tiae a reaponae reachea the 

point of origin to the tiae it reachea ita point of 
iapact. 

Therefore the total reoponae ia 

6 

" i«l ^ 


Alternative deaigna are expected to be evaluated on an equ4|il 
baaia. Equal baaia coapariaon ia feaaible if all aegaenta of a 
certain aeasure are coaplete. If in an alternative ^ a aegaent la 
aiaaing (e<.g«) 41*5, in the reaponae tiae meaaure) that alternative 
aeaaure cannot juatifiably be coapared with the aaae aeasure of 
another alternative whose aegaenta are coaplete* In that case> a 
**dunny" insertion is to be added to the alternative that has 
incoaplete segments to make the equal basis criterion applicable. 

As it is not expected that all alternatives will address all 
response tiae segments) a number of ’’reasonable" response time 
insertions can be added wherever needed. ’’Reasonable" oieans an 
estimate taken from the present method of performing a corresponding 
function. For example) if alternative 1 ia designed to modify 
segment of the response time) while alternative 2 modifies 
segments AT2» 4T3) and dT4> then values estimated for AT2 and 

of response time segments whould be inserted in alternative l\\ in 
order for the two alternatives to have equal numbers of segments.!! If 
a third alternative is considered which addresses the modification of 
all response time segments) then alternative 1 will need 5 insertions 
(ATj) dl2) AT^) AT5) and ATg) and alternative 2 will need 
three insertions (AT|) AT^) and ATg) equal in value to AT|) 
and ATg and ATg of alternative 2. Figures 3-3 and 3-4 summarize 
this discussion. 




<I 


AT, 


<1 

ALTERNATIVE 1 



V 




ALTERNATIVE 2 


V 


V 



ALTERNATIVES 

V 

V 


V 

V 










Figure 3-3. Response Time Segments of Three Alternatives 
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Figure 3 -A. Response Time Insertions Needed for Each Alternative 


In case all alternatives address the modification of the same 
response time segements, then those segments that are not relevant to 
the analysis should be dropped. For example, if dT3 is the only 
segment to be modified in all alternatives, then, dTx, dT2« 

and dTg will not be considered and therefore dropped (i 
from the computation. 

It shouj.d be noted that response time is a variable dependent on 
throughput, service times , and number of facilities. 

e. Methodology of Application . Different documents/ 
transactions will be processed differently^^ will have different 
response times, and flow along different paths on their way out of the 
system. 

The variation in resppnse times could be attributed to 
variations in the document parameters, variation among servers, 
variation within each server, and/or interaetion between two or more 
of these factors . In this case it is reasonable to aissume that 
processing tiaie at each work station is an independent random 
variable, xj. Each one of these random variables could be 
represented by a certain distribution with a mean of the x^ and a 
variance of <t2. 

Irrespective of the type of distribution functions, the sum of 
processing times of all work stations (including transportation and 
transmission) would have a limiting distribution. According to the 
Central Limit Theorem that limiting distribution is normal* with a 
mean equal to: 


. This is approximately true for large n» 


*as n approaches infinity 



and a variance equal tot 



0 



where n la the number of work atationit 


If volume and priority by document type is introduced, the mean 
reaponse time would be equal to; 


4T 


t .jQj 
j-1 J ^ 


(Equation 6) 


and the variance would be equal to; 


m , 

t .jQj 

j-1 ^ ^ 


(Equation 7) 


where aj ■ document type priority 

Qi ■ volume by document type „ 

Xj - responae time by document type 
j - document type 

m - number of document type 

/) 

Responae time samples (>2n; n is the number of stations) can be 
used to generate cumulative distribution function values in order to 
determine the overall response time of processing eacti card/document 
with probability P. Figure 3-5 illustrates the cumulative distribution i 
function of a noirmally distributed response time. 


Iv 
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Figure 3-5, Cumulative Denaity Function of a Normally 
Distributed Response Time 


In order to test alternative design response times using the FBI 
guidelines^ the following set of equations may be used: 

F(K) - P(dT;gK) 

F(8) » P(dT$8 hrs) - 0.95 

F(48) •« P(dT:S48 hrs) ■ 0*99 

F(96) - P(4T^96) - 0*999 

where F(K) is the cumulative distribution function and 4T is a random 
variable representing response time. 


2s Reliability* 

a. Definition . Component reliability is the probability that 
the component will function without failure over a specified period of 
tisw, t. 

R(t) - P(T>t) 

where T is the time to failure of the component R(t) is the 
reliability function. ^ 


*Although software reliability is of paramount significance, the nature f 
of the feasibility study limits the use of the term to hardware reliability 



Pi*cuiiiion » For 6hs purpose of this study ^ the tiise to 
failure is considered a continuous rando* variable described by an 
exponential distribution with a constant falluro rate»A* Thus the 
reliability function can be put into the following for*! 


R(t^ ■ e“At (Equation 8) 


The exponential failure distribution iaplies that the 
probability of failure is independent of past history and so long as a 
coisponent is still functioning it is "as good as new." 

For the purpose of this studyi statistical independence is 
assuned for all conponents. 

For a system structure of two or more components (subsystems) 
functioning independently of each other/t the reliability of the system 
depends on two factors: 

(1) The reliability of each conponent. 

(2) The configuration of system components. 

The following generalized configurations are considered as data 
from which particular system reliabilities can be computed* 

1) Components in Series . Consider R£ to be the reliability 
of ith component, Ci, in a series component system as shown in 
Figure 3-6. 

In order for the system to function, each one of the components 
must be operating successfully. Therefore the system reliability, 

, is: 



n 

77 

i-1 


Ai 


SYSTEM 



Figure 3-6. Components in Series System 



or 



(Equofcion 9) 


2) C^ponontr in Parallel . In order for the eye ten to 
function^ at leaet one coaponent auit P^e operating aucceaafullyi aee 
Figure 3-7 i then the ayatea reliability » Rg, ia: 


R. •• 1 


R, - I 


probability the ayatem ia in a failed atete 
n 


n (1 - R.) 
i-t ^ 


(Equation 10) 

R - 1 - /7 (I - e “^i*') 

■ i-l 

If all the componenta have equal reliabilities for all i« then 
Equation R beeoaes; 


R^ - 1 - (1 - 


Figure 3-8 ia an illustration of aeries and parallel structure 
syateiB reliabilities* 


SYSTEM 



Figure 3-7* Conponenta in Parallel Syatem 
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COMPONENT REUABILITY 

Figure 3-8. Series and Parallel Component System Reliabilities 


r 

% 


3) Components in Combined Series/Parallel Cortfiguratjon 
(Component Redundancies In this configuration, the system will 
function if at least one component is operating successfully in each 
stage when needed. See Figure 3-9. 


The system reliability, Rg, is:,. 


« J 


k 

77 


1 - 


n 

77 (1 

i?=l 


- R. 


1 » J 


(Equation 11) 
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Figure 3-9 • Components in Combined Series/Parallel Configuration 


When components are identical within a stage) then system 
reliability becomes : 


1 - d"- Rj)" 


Figure 3-10 shows that the system reliability as a function of 
component reliability is represented by a reverse curve. The shape of 
the curve suggests the impact of component reliability) number of 
stageS) and redundancy on the reliability of a composite structure 
system. 

4) Components in Combined Parallel/Series Configuration 
(System Redundancy) . In this configuration. Figure 3-11) the system 
will function if any set of the series components is operating 
successfully. The system reliability is.: 


R 


k 

77 

3-1 



n 

1 - 77 

i-1 


k 

1 - 77 R_. 
j-1 


If J 



(Equation 12) 



COMPONENT RELIABILITY 


Figure 3-10. Composite Structure System Reliability 

f . . 

When components are identical within a series set, then system 
reliability becomes: 

k n 

R- 1 - 1 - 77 R, 

I J . 

When all components throughout the system sets are equal, then 
the system reliability is: V 
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Figure 3-11. Components in Combined Parallel/Series Configuration 


The difference between system reliability of configurations 3-9 
and 3-11 indicates that redundancy at the component level is more 
effective than redundancy at the system level. Figure 3-12 is an 
illustration of this remark using a 4-component system in each case. 


c. Application . Reliability as a measure of effectiveness 
may be included in the evaluation process for two reasons: 

(1) To compare reliability of alternative system designs with 
that of the functional requirements, if any. 

(2) To assess the adequacy of maintenatitse strategy and 
maintenance cost projections of each alternative. 

The reliability computation can be handled by using a system 
modular decomposition approach as follows: 

A system is decomposed into its major subsystems. Each major 
subsystem is decomposed into subsystems. Each subsystem is decomposed 
into components. This process will continue to the level of parts 
specified by each respective design. From this information, the 
reliability of each component will be determined. Then the 
reliability of each subsystem will be computed. The process will 
continue up to the system level. 


i: 
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ji 3. Availability 

i ■ ■ 

a* Definition . Availability is the ratio of the time the 
1 required elements of the system are operational to the total time the 

j system or any of its elements are expected to be operational. 

For one simple production unit, availability is equal to: 


I 

J- 



MTBF 

MTBF + MTTR 


where MTBF is the mean time between failures, in hours, and MTTR is 
the mean time to restore, in hours. The MTTR includes the mean time 
to request service, the mean time to respond, the mean time to travel 
fhe mean time to debug, the mean time to repair, and all other mean 
times that will keep that unit from entering production assembly line 
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Three system availability configurations are considered; 

'.J 

(1) For series structure with one failed component » 

functioning components suspended their operation during 
the time thst component is unavailable. System 
availability, Ag, is given by; 


R. (Equation 13) 

E f: 

-•-I X 


«rhere R ■ MTTR, F ■ MTBF for component i. 

(2) For a parallel structure system, the system availability 


n 

A • 1 - /7(1 - A.) (Equation 14) 

* i-1 ^ 


Where Aj^ is the component availability. 

(3) For a series/parallel structure, the system availability 
is; 



k 

• n 

j-1 


1 


n 


77 (1 
i«l 



(Equation 15) 


where i stands for components in a stage and j stands for stages in a 
system. 

This study recognizes the fact that failures vary from "hard 
failure", where system operation completely halts, to "soft failure" 
«diere the system continues to operate but in a degraded mode. However, 
in this document component "hard failure" is assumed. Depending on 
the system configuration and hierarchy, the previous assumption may or 
may not lead to complete halt of system operations. 

b. Application Methodology . Availability, unlike reliability, 
relates directly to the productivity of the system. The most 
significant implication of availability is the fact that if a component 
is not functioning .^t will either add to the workload of other 
componehts and increase the queue sizes or completely halt the system 
production during the time it is unavailable. 


The unavailability duration is what nakea availability 
conaideration a critical aeaaure of effectiveneaa for production 
ayatena auch aa thoae in the Identification Diviaion operationa. 

The availability cooq>utation could be handled by syatem 
decompoaition in a manner aimilar to reliability calculationa . The 
atructure in Figure 3-13 ia an example. It ia a tree atructure ayatem 
where the failure of one or Vj ia not aasociated with the 
failure of other M^a or Vja» but“^the failure of component C or E 
will diaable the ayatem input-output flow for &11 intenta and 
purpoaea. Therefore, the ayatem will be unavailable if all Vja, all 
M^a, complementa of Vja and Nj^a, C, or E are unavailable. Then 
the ayatem availability of the given atructure ia: 



m 

77 

i-1 



77 (1 - 

j-1 _ 



(Equation 16) 


-a 



Figure 3-13. Example of Syatem Decompoaition 
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4. Utilization 

i. > 


a. Definition . In a flow syitem network^ utilization is the 
ratio of the rate at which a workload is applied to a facility to the 
maximum rate at which the facility can proceaa this workload. 


b. Discuaaion . In a fingerprint identification facility ^ a 
server may be busy part time, most of the time, or all the time. If 
he is busy all the time, we say his utilization is 1.0. But if he is 
busy 70% of the time and idle waiting for another fingerprint the 
remaining 30% of the time, we say his utilization is 0.7. Similarly 
any facility, be it a computer, a terminal operator, or a communica- 
tion controller, is regarded to be operating at a certain utilization 
rate . ^ , 

In production systems one ‘of the objectives is to maximize 
utilization of operators and machines in order to getithe most 
production at the least possible cost. Although it ii desirable tp 
achieve a utilization of 1.0 for every facility, the side effects, as 
a result of random service times, are large queues and long delays as 
utilization approaches 1.0. 

Utilization is dependent on two variables, the arrival rate E(n) 

and the service time E(t ), so that: 

s 


P ■ E(n) X E(t_) 0 < /) < 1 (Equation 17) 

9 

for a single-server facility and 

P * Mul X E(t ) 0 < 1 (Equation 18) 

in s 


for a multi-server facility where p is called the facility utilization 
and m is the number of servers. 

For the purpose of illustration, assume we have the following 
queueing system described by the notation: 

M/M/m/oo/FCFS 

Where the first and second letters M denote that interarrival 
time and service time are exponentially distributed, the third 
descriptor stands for the number of servers. The fourth descriptor 
denotes that no restriction On queue size is imposed and the fifth 
descriptor indicates that the discipline is based on first come first 
served. 

i 
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The steady state average wait time in the system (i.f/.» average 
wait time in queue plus mean service time in the facility) is 
represented hy the following algorithm: 


B E(t ) 

E(t^) ■ - “"iZp" ^ (Equation 19) 


and the steady state average queue length in the system (i«e.^ units 
waiting for the service plus those being serviced) is represented by 
the following algorithm: 


(Equation 20) 


B probability that all servers are busy (see Appendix A) 
m ■ number of servers " 

Note that B * P when m ■ 1 . 

A plot of average waiting time in the system in multiples of 
service times versus facility utilization is given in Figure 3-lA> A , 
plot of average queue size divided by number of servers versus 
facility utilization is given in Figure 3-15. 

In both cases when the facility utilization is higher than 
0. Sty the average waiting time and the queue size start accelerating 
rapidly for small increases in workload Volumes. This indicates that 
Q point of saturation is imminent if the workload volume increases 
either steadily or through an impulse. For example, a delta increase 
of input volume, dvy will increase queue size by: 

X E (t ) X dv (Equation 21) 
s 


Table 3-2 lists queue length increases Jfor various values of 
utilization of a single-server facility. It clearly indicates that 
marginal system designs with high utilization could adversely impact 
throughput and response times. 

Another important aspect of utilization lies in he area of 
feedback and control lo^ic of production systems. 

If the system logic were to simulate the movement of documents 
and data throughout the system for the purpose of control, the 
validity of the,f.iodel and hence the effectiveness of the control 


1 + 


P(2 ^ P) 
(1 -P)' 


E(q) ■ B + mp 

where 
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Figure 3-15. 




Average Queue Size Versus Facility Utilization 
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■trategy could b« advaciely iippacCad for utiliaafion factora e;;(ceeding 
1»0 (fchia meant departurei and arrival a of tranaacCiona are not 
balanced). Figure 3-16, Gate 2 illuacratoa a condition where the 
utilisation factor ia 1.0. If thia were to happen unchecked > then the 
ayatem network will be aubject to perturbationa as a reault of loos of 
tracking and orderly control of tranaactiona. 


Table 3-2, Queue Length Xncreaaea for Various Values of Utilistion 


Queue Length Queue Length Increase as 

Utilization Increase Multiple of Base Case^ 


0.95 

400 

X E(tg) 

X dv 

67 

0.90 

100 

X E(tg) 

X dv 

17 

0.80 

26 

X E(tg) 

X dv 

4 

0.70 

12 

X E(tg) 

X dv 

2 

0.60 

6 

X E(tg) 

X dv 

1 


;7 


ei.e., with 0.6 utilization. 


Another aspect to utilization in a flow system network is 
storage capacity between nodes or work stations. If queue sizes can 
grow without restriction, then capacity has no bearing on 
utilization. A close look at^ storage capacities may, in all 
probability, reveal that restrictions are being imposed on queue 
sizes at certain points of the system network. The impact could be 
explained in the following scenario: 

The system is made up of a aeries of stages. A calling unit 
can leave stage j if the storage capacity between stage j and j+l is 
not full. But if it is full, the calling unit will remain in stage j 
and block ahy other calling unit from entering that stage. The 
result is a decreade in the utilization of stage j. If the blocking 
effect is allowed to happen in every stage, then the first stage will 
be blocked most frequently, and the one before the last will be 
blocked the least. If service times become insignificant compared to 
delay time due to blocking, then utilization of the network will be 
determined by the utilization of the first stage. 
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CASE 1 E(n),<E(») o^p<I 

II ' ® 1 

E(n)|" E(o)2 + E(n)2 
CASE 2 E(n) | < E(^^) ^ p > 1 
E(n)| > E(n)2 + E(n)g 


Figure 3-'i6, Impact of Utilization on Feedback 
and Control Strategy 


5. Accuracy and Completeneaa 

a. Accuracy . This meaaure aeeks to detennine the level of 
correct or incorrect identifications made at the system level. The 
numerics can be expressed in percent or probability of occurence. Two 
parameters of the accuracy measure are: 

(1) False drop rate: Percentage level of incorrect subjects 

returned by the system. 

(2) Miss rate; Percentage level of subjects not identified by 
the system although they are in files. 

The above two parameters are composites of accuracy levels of 
various subsystems; i.e., the overall result of combinations of name 
search i fingerprint search > and verification to generate final results. 

Subsystem accuracy level may be used for comparison purposes 
among alternatives > but it cannot be used as an indicator for overall 
system accuracy level since it is only one of several components. 

An experimental design followed by analysis will be required in 
Order to address adequately the subject of system accuracy. 



The eccuricy of file updetef if coniidered e fubiel: of the felte 
drop end eccurecy retei, Evaluerorr My look for Mthodology used in 
insuring accurate fi^le update for various alternative designs. 

b. Coapleteness . This Measure points to the design approach 
taken to insure that all available inforMation concerning a subject 
abailable to the Identification Division is included in the files in a 
tisMsly Mnner and can be outputted on an Identification Division form 
when needed. 
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APPENDIX A (_J 
PROBABILITY OF BUSY SERVERS 


Values of Probability that AU< m Server* Are Busy, B 

Facility 

Utiliza' Number of Servers, m 

tion 



m»l 

in " 2 

m *3 

in “4 

m *5 

m »6 

ni “7 

1 m »8 

m »9 

0 

1 

0.00 

0.000 

0,000 

0.000 

0,000 

0.000 

0.000 

0.000 

0.000 

0,000 

0.000 

0.02 

0.020 

0.001 

0,000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.04 

0.040 

0,003 

0.000 

0.000 

0,000 

0.000 

0.000 

0,000 

0.000 

0.000 

0.06 

0.060 

0,007 

0.001 

0.000 

0,000 

0.000 

0.000 

0,000 

0.000 

0.000 

0.08 

0.080 

0.012 

0.002 

0.000 

0,000 

0.000 

Q .000 

0,000 

0.000 

0.000 

0.10 

0.100 

0,018 

0,004 

0.001 

0,000 

0.000 

0.000 

0.000 

0,000 

0,000 

0.12 

0.120 

0,026 

0.006 

0.002 

0,000 

0.000 

0,000 

0,000 

0.000 

0.000 

0.14 

0.140 

0.034 

0,009 

0.003 

0,001 

0.000 

0.000 

0,000 

0.000 

0.000 

0.16 

0.160 

0,044 

0.014 

0,004 

0,001 

b.ooo 

0.000 

0.000 

0.000 

0,000 

0.18 

0.180 

0.055 

0.019 

0,007 

0,002 

0.001 

0.000 

0.000 

0.000 

0,000 

0.20 

0,200 

0.067 

0.025 

0.010 

0,004 

0.002 

0,001 

0.000 

0,000 

0.000 

0.22 

0,220 

0,079 

0.032 

0.013 

0.006 

0,003 

0.001 

0.001 

0.000 

0,000 

0.24 

0.240 

0.093 

0.040 

0,018 

0.008 

0.004 

0.002 

0,001 

0,000 

0,000 

0.26 

0.260 

0.107 

0.049 

0.023 

0.011 

0,006 

0.003 

0.001 

0,001 

0.000 

0.28 

0.280 

0.122 

0,059 

0,030 

0.015 

0,008 

0.004 

0,002 

0.001 

0,001 

0.30 

0.300 

0.138 

0.070 

0.037 

0.020 

0,011 

0.006 

0,004 

0.002 

0.001 

0,32 

0.320 

0.155 

0.082 

0.046 

0,026 

0.015 

0.009 

0.005 

0.003 

0.002 

0.34 

0.340 

0.173 

0.095 

0.05 S 

0.033 

0.020 

0,012 

0.007 

0,005 

0.003 

0.36 

0.360 

0.191 

0.110 

0.066 

0,040 

0,025 

0.016 

0.010 

0.007 

0.004 

0.38 

0.380 

0,209 

0.125 

0.078 

0.049 

0.032 

0.021 

0.014 

0.009 

0,006 

0.40 

0.400 

0.229 

0.141 

0.091 

0.060 

0,040 

0.027 

0.018 

0.013 

O .009 

0.42 

0.420 

0.248 

0.158 

0.105 

0.071 

0.049 

0.034 

0,024 

0.017 

0,012 

0.44 

0.440 

0.269 

0,177 

0.120 

0.084 

0,059 

0.043 

0,031 

0,022 

0,016 

0.46 

0.460 

0,290 

0.196 

0.137 

0,098 

0.071 

0.052 

0.039 

0,029 

0.022 

0,48 

0.480 

0,311 

0.216 

0,155 

0,114 

0.084 

0.064 

0,048 

0.037 

0,028 

0.50 

0.500 

0.333 

0,237 

0.174 

0,130 

0,099 

0.076 

0,059 

0.046 

0.036 

0.52 

0,5120 

0.356 

0.259 

0.194 

0,149 

0,115 

0,090 

0.072 

0.057 

0,046 

0.54 

0.540 

0.379 

0.281 

0,216 

0.168 

0,133 

0,106 

0,086 

0,069 

0.057 

0.56 

0.560 

0.402 

0.305 

0.238 

0.190 

0.153 

0.124 

0.1 02 

0,084 

0,069 

0.58 

0/580 

0.426 

0.330 

0,262 

0.212 

0.174 

0.144 

0.120 

0,100 

0.084 

0.60 

0/600 

0.450 

0.355 

0.287 

0.236 

0,197 

0.165 

0.140 

0.119 

0.101 

0.62 

4620 

0.475 

0.381 

0.313 

0.262 

0,221 

6>188 

0.161 

0.139 

0.12 G 

0.64 

fom 

0.500 

0.408 

0.340 

0.289 

0.247 

0.213 

0,185 

0.162 

0.142 

0.66 

0.660 

0.525 

0.435 

0.369 

0.317 

0,275 

0.241 

0,212 

0.187 

0.166 

0.6 S 

0,680 

0.550 

0.463 

0.398 

0.347 

0.305 

0.270 

0.240 

0.215 

0.193 

0,70 

0.700 

0.576 

0.492 

0.429 

0.378 

0.336 

0.301 

0.271 

0.245 

0.222 

0.72 

0,720 

0.603 

0.522 

0,460 

0.410 

0.369 

0.334 

0.303 

0.277 

0.254 

0,74 

0*740 

0.629 

0.552 

0.493 

0.444 

0.404 

0,369 

0,339 

0.312 

0.288 

0.76 

0.760 

0.656 

0.583 

0.526 

0.480 

0.440 

0.406 

0.376 

0,349 

0,326 

0.78 

0.780 

0.684 

0,615 

0.561 

0.516 

0.478 

0.445 

0,416 

0,390 

0.366 

0.80 

0.800 

0.711 

0,647 

0.596 

0,554 

0.518 

0.486 

0,458 

0.432 

0.409 

0.82 

0,820 

0,738 

0,680 

0.633 

0,593 

0.559 

0.529 

0.502 

0,478 

0.455 

0.84 

0,840 

0.767 

0.713 

0.670 

0,634 

0.602 

0,574 

0.548 

0,525 

0.504 

0.86 

0.860 

0.795 

0,747 

0.709 

0.675 

0.646 

0.621 

0.597 

0.576 

0.556 

0.88 

0.880 

0.824 

0.782 

0*748 

0.718 

0,693 

0.669 

0,648 

0.629 

0.611 

0.90 

0.900 

0.853 

0.817 

0,788 

0,762 

0,740 

0.720 

0,702 

0.687 

0.669 

0.92 

0.920 

0.882 

0.853 

0.829 

0,808 

0.789 

0,772 

0,757 

0.743 

0.729 

0.94 

0,940 

0.911 

0.889 

0,870 

0.854 

0.840 

0.827 

0.815 

0.803 

0.793 

0.96 

0,960 

0.940 , 

0.925 

0,913 

0.902 

0.892 

0,883 

0.874 

0.866 

a ?59 

0.98 

0.980 

0.970 

0,962 

0.956 

0.950 

0.945 

0,940 

0,936 

0,932 

0.928 
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APPENDIX B 
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ACS 

APRS 

AHU 

AIDS 

ANS 

ATS 

ATSPS 

AUTOCOR 

AUTORESP 

A&R 

BER 

BLO 

CCA 

CCH 

CCN 

CCNR 

CCR 

CIR 

CLASS -A 

CLASS-B 

^LASS-^C 

C^LCK 

CNR 

COA 


II ACRONYMS 

AutomaCed Classification Systeai 

Autonated Fingerprint Pleader Systea 

Anti-'Ualation Underlayer 

Automated Identification Division System 

Automated Name Search 

Automated Technical Search 

Automated Technical Search Pilot System 

Automated Correspondence Station (part of AIDS) 

Automated Response Generation (part of AIDS) 

Automation and Research Section of Identification' 
Division 

Bit Error Rates 

Blocking Out 

Computerized Contributor Abbreviated Name 
Computerized Criminal History (part of NCIC) 
Computerized Criminal'^^Name 

Computerized Criminal Name and Record (part of AIDS) 
Computerized Criminal (Arrest) Record (part of AIDS) 
(Computerized Ident Response File (part of AIDS) 

Class ification-’A 

Classification-'B 

Classif ication-C 

Classification Check 

Computerized Non-Ident Response File 

Cutoff Age 

Central Processing Unit 
" B-1' 


CPU 


0 


CRS 

CRT 

CSORT 

DATE STP 

DBMS 

DEDS 

DENT 

DENT-A 

DENT-B 

DOA 

DOB 

ECL 

EMI 

ENC 

ENCDOC 

ENCK 

ENDOCK 

ERR 

EYE 

FBI 

FEP 

FIFO 

FLAB 

FLOAD 

FPC 

FPCS , 

,/ 

f/P 


Conputerised Record Sent File (part of AIDS) 

Cathode Ray Tube 
Centerline Sort 
Date Stamp t Count and Log 
Data Base Management System 

Data Entry and Display Subsystem (part of AIDS III) 

Data Entry 

Data Entry-Cards 

Data Entry-Documents 

Date of Arrest (on f/p card) 

Date of Birth (on f/p card) 

Emitter Coupled Logic 
Electromagnetic Interference 
Encode Input Data-Cards 
Encode Input Data-Documents 
Encode Check-Cards 
Encode Check-Documents 
flpdate Error File 
Color of Eyes (on f/p card) 

Federal Bureau of Investigation 

Front End Processor 

First-In-First-Out 

Film Lab Processing/Computer 

Film Load 

Fingerprint Classification 

Fingerprint Correspondence Section of the Identification 
Division 

Fingerprint 
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GDdHS 

CEO 

GPSS 

HAl 

HCT 

IBM 

ICI 

ICRQ 

ICS 

lev 

ID, I.D. 
IDENT 
JPL 
KIPS 

LEAA 

MAIL 

MFILM 

MIPS 

MMF 

MOE 

MTBF 

MTR 

MTTR 

NAM 

NASA 

NCIC 


General Purpose Data Bsae Management System 
Geographic Location (on £/p card) 

General Purpose Simulation System 
Color of Hair (on f/p card) 

Height (on £/p card) 

International Business Machines Corporation 
Image Comparison Identification 
Image Comparison Request 

Image Comparison Subsystem (part of AIDS III, actually 
used for image retrieval for manual comparison) 

Image Comparison Verification 

Identification Division 

Identification 

Jet Propulsion Laboratory 

Thousands of Instructions per Second (as executed by a 
computer) 

Law Enforcement Assistance Agency 
Open Mail and Sort 
Image Capture Microfilm 

Millions of Instructions per Second (as executed by a 
computer) 

Minutiae Master File 
Measures of Effectiveness 
Mean Time Between Failures 
Master Transaction Record 
Mean Time to Repair 
Name (on f/p card) 

National Aeronautics and Space Administration 
National Crime Information Center 
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NCR 

OCA 

OCR 

0MB 

ORI 

PCN 

PICS 

PMT 

POB 

QC 

QUERY 

RAC 

READ 

RFI 

RH 

RVF 

SACS 

SAR 

SEAR 

SEX 

SID 

SKN 

SOC 

SPM 

SS 

SSM 

SSRG 


National Cash Register Company 

Local Identification Number (on £/p card) 

Optical Character Recognition 
Office of Management and Budget 

Originating Agency Identification Number (on f/p card) 
Process Control Number 

PCN and Image Capture Subsystem (part of AIDS III) 

Photomultiplier Tubes 

Place of Birth (on f/p card) 

Quality Control 
On-Line Query 
Race (on f/p card) 

Quality Control Check. Read , Annotate 

Radio Frequency Interference 

/:> 

iitelative Humidity 
Ridge Valley Filter 
Semi-Automatic Classification System 
Semi-Automatic Fingerprint Reader 
Search Review 

Reported Sex of a Subject (on f/p card) 

State Identification Number 
Skin Tone (on f/p card) 

Social Security Number (on f/p card) 

Search Processor Module 

System Supervisor Subsystem (part of AIDS III) 

Subject Search Module 

Subject Search and Response Generation Subsystem (part of 
AIDS III) 


TDFA 


TFC 

TR 

TRC 

TSS 

TTL 

VDENT-A 

VDENT-B 

VLSI 

WAND 



Top Down Functional Analyais 
Technical File Conversion 
Transaction Record 
Transaction Control File 

Technical Search Subsystem (part of AIDS III) 

Transistor - Transistor Logic 

Verify Data Entry-Cards 

Verify Data Entry-Documents 

Very Large Scale Integration 

Wand Out of System 
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